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ABSTRACT 

Patients with motor control difficulties often “type” on a 
computer using a switch keyboard to guide a scanning cursor 
to text elements. We show how to optimize some parts of 
the design of switch keyboards by casting the design problem 
as mixed integer programming. A new algorithm to hnd an 
optimized design solution is approximately 3600 times faster 
than a previous algorithm, which was also susceptible to 
hnding a non-optimal solution. The optimization requires a 
model of the probability of an entry error, and we show how 
to build such a model from experimental data. Example 
optimized keyboards are demonstrated. 

Categories and Subject Descriptors 

J.4 [SOCIAL AND BEHAVIORAL SCIENCES]: Be- 

havior Informatics; 1.6 [SIMULATION AND MODEL¬ 
ING): [Model Development] 

General Terms 

Design, Human Factors, Experimentation 

Keywords 

“Locked-in” Patients, Switch Keyboard, Mixed Integer Pro¬ 
gramming 

1. INTRODUCTION 

Some patients with spinal cord or brain injury lose motor 
control skills, even when they maintain their cognitive func¬ 
tions. In extreme cases of “locked-in” syndrome, patients are 
almost entirely paralyzed but remain conscious and need a 
means of communicating their thoughts. One method of 
communication is a switch keyboard with a scanning cursor 
that traverses a virtual keyboard. A binary switch triggered 
by the patient (e.g., with an eye blink or a puff of air) guides 
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the cursor to a character that is then typed on the computer. 
Such typing is very slow (often measured in characters per 
minute rather than words per minute), and our goal is to 
make the process more efficient by identifying how to place 
characters on the keyboard in a way that allows for fast typ¬ 
ing with few errors. 

Francis and Johnson proposed that character placement 
could be treated as cost minimization that traded off speed 
and accuracy. Calculating these terms required a corpus of 
the kind of text that the patient would enter (e.g., poetry vs. 
HTML code), a model of errors, and an allowable average 
error rate. With such information, a hill-climbing algorithm 
could identify the cursor duration and key placements that 
minimized entry time. 


2. MIXED INTEGER PROGRAMMING 

Here we show how to reframe the character placement 
problem as a mixed integer programming problem. A key¬ 
board includes a set X of keys, and the cursor path scans 
across the rows, JT", and columns, /C, as guided by the user. 
Denote fi as the entry frequency of character z in a given 
text of characters. Set Xijk equal to 1 if character i is as¬ 
signed to row j and column k, and 0 otherwise. Suppose the 
cursor moves across rows until the user triggers the switch 
to select a desired row. The cursor then moves across keys 
in the selected row until the user triggers the switch to select 
a particular key. Under such a cursor movement plan, the 
time to input a character at position j, k is tjk = D(j + /c), 
where D is the duration of the cursor for each step. Assume 
Pjk{D) is the proportion of errors when attempting to guide 
the cursor to position j, k, and that a user identifies an ac¬ 
ceptable error rate of e. 

Some characters are best assigned to hxed positions on the 
keyboard. For example, the numbers of 0,1,..., 9 are tradi¬ 
tionally grouped together in keyboard layout designs, so we 
can assign them as follows to the bottom of the keyboard: 

5 = fc)|(55, 7, 7), (56, 7,8), (57,8,1), (58, 8, 2), (59,8,3), 

(60,8,4), (61,8, 5), (62,8,6), (63, 8, 7), (64,8,8)} 

where i is the key index (in our example, keys 55-64 repre¬ 
sent the numbers 0-9). Then the problem of minimizing the 
entry time is as follows, given corpus size n, D, e, and /i, 

minimize Ct ^ EXE fitjkXijk (1^) 

iex jej keic 


subject to the following constraints: 


" F F F fiPik{D)xijk < e 

(lb) 

i G j G y k^KL 


Xijk = 1, for i G X; 

(Ic) 

jej keK, 


'^Xijk = 1, for j e J;k e JC; 

(Id) 



Xs = 1, for s G <S. 

(le) 


Equations (la) and (lb) are the definitions of the time cost 
and the error cost. Constraint (lb) ensures that the er¬ 
ror rate is not greater than the given acceptable threshold. 
Constraints ( p^ ensure that each key can be only assigned 
to one position. Constraints ensure that each position 
can only contain one key. Constraints ( p!^ assign the num¬ 
ber keys to their fixed positions. 

The problem can be solved by using a Gurobi Optimizer with 
standard techniques that both guarantee a globally optimal 
solution (if it exists) and requires only about 3 seconds for a 
desktop computer. In contrast, the hill climbing algorithm 
on the same computer required approximately 3 hours to 
find a (not necessarily global) solution. 


3. MODELING ERROR PROBABILITIES 

For the optimized keyboard to be useful, the pjk{D) terms 
need to accurately reflect the error probabilities of a user. 
The task is to time a switch action to guide the cursor to 
a target key. We suspected that the timing of the switch 
action could be modeled with a Gamma distribution: X ~ 
r(Av,^), where k is a, shape parameter and ^ is a scale pa¬ 
rameter. The density function is denoted as f{x;K,0) = 
for > 0 where r(j;) is the gamma 

function evaluated at x, which is the elapsed time. 

For each row in the keyboard and each cursor duration, we 
estimated the distribution parameters using data from Fran¬ 
cis and Johnson [^. Their data indicated whether a switch 
response was generated early, correctly, or miss. A Gamma 
distribution can produce similar probabilities by consider¬ 
ing the area under the cdf prior to x — Dj^ where j is the 
target row, between Dj and D{j + 1), and after D{j + 1). 
We used a Nelder-Mead optimization method to estimate 
the distribution parameters. Figure shows a scattergram 
of the observed error proportions against the model fit. 
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Figure 1: Scattergram of observed error probability 
and model fit 


elapsed time and cursor duration. They suggest systematic 
patterns that will be modelled in future work in the Gamma 
distributions. 
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Figure 2: /^-parameter Figure 3: ^-parameter 


4. OPTIMIZATION RESULTS 

Figurej^shows two optimal keyboards that trade off speed 
and accuracy for a fixed cursor duration D = 0.35 seconds. 
The keyboard on the left used e = 0.30 and so emphasizes 
character placement to minimize the error rate when enter¬ 
ing a given text corpus (a set of famous quotes). The key¬ 
board on the right used e = 0.50 and so emphasizes character 
placement to minimize the time of entry. 



(a) e = 0.30 


(b) e = 0.50 


Figure 4: Two optimized keyboards 


5. CONCLUSIONS 

We identified how to solve the problem of optimal char¬ 
acter placement on a switch keyboard using mixed integer 
programming. The algorithm is orders of magnitude faster 
than a previous approach and will allow for consideration of 
many other keyboard properties (e.g., various cursor paths, 
different switch devices, shortcuts). The algorithm requires 
an accurate model of error rates, which must be based on hu¬ 
man data. We described a model that matches the observed 
data and has promise for future investigations. 

6. ACKNOWLEDGMENTS 

This work was supported by a Project Development Team 
within the ICTSI NIH/NCRR Grant Number RR025761. 


Figures and plot the k , and 0 estimates as a function of 


7. REFERENCES 















































[1] G. Francis and E. Johnson. Speed-accuracy tradeoffs in 
specialized keyboards. International Journal of 
Human-Computer Studies, 69(7-8):526-538, 2011. 



